Uploading Large Files with Git LFS

This page is for researchers who need to put models or other large files (.nemo, .onnx, .bin, n-gram LMs and so on) into a repo on Gitea. Downloading works from anywhere with no extra setup. Uploading files over 100MB needs a small one-off change, described below.

Why uploads need extra setup

Gitea has two addresses, and they reach the server by different routes:

Address	Route	Upload limit	Who can use it over HTTPS
gitea.abair.ie	Through Cloudflare	100MB per file	Anyone
gitea-direct.abair.ie	Straight to the server	None	TCD network only (on campus, or on the TCD VPN)

Git LFS uploads each file in one piece over HTTPS. By default it sends them to gitea.abair.ie, even if you cloned using gitea-direct, so any file over 100MB is rejected by Cloudflare. The fix is to tell Git LFS to upload to gitea-direct.abair.ie instead. That only works from a TCD address.

In short

To upload large files you need to be on the TCD network or VPN, and point Git LFS at gitea-direct.abair.ie (step 4 below). Nothing else about how you use git changes.

What happens in each situation

You are…	Git LFS uploads to…	Files under 100MB	Files over 100MB
On TCD network or VPN	gitea-direct.abair.ie (set up as below)	✅ Works	✅ Works
On TCD network or VPN	gitea.abair.ie (the default)	✅ Works	❌ Fails with HTTP 413
Outside TCD	gitea.abair.ie (the default)	✅ Works	❌ Fails with HTTP 413
Outside TCD	gitea-direct.abair.ie	❌ Hangs, then times out	❌ Hangs, then times out

Normal git commands (clone, pull, push of code) over SSH work from anywhere.

Before you start

You need:

• a Gitea account with two-factor authentication set up, and an SSH key added under Settings → SSH / GPG Keys;
• write access to the repo. Model repos live in the ABAIR organisation; ask an admin to create the repo or give you access if you can't push to it;
• git-lfs installed. Check with git lfs version. If it's missing:

  sudo apt install git-lfs     # Debian / Ubuntu
  brew install git-lfs         # macOS
  git lfs install              # once per machine, after installing

One-off setup

1. Create a Gitea access token

Git LFS logs in to gitea-direct.abair.ie with a token, not your SSH key.

1. In Gitea, go to Settings → Applications → Generate New Token.
2. Give it a name, such as lfs-uploads-<your machine>.
3. Under Select permissions, set repository to Read and Write. Leave everything else as No Access.
4. Click Generate Token and copy it. Gitea only shows it once.

2. Save the token in ~/.netrc

Add these lines to ~/.netrc in your home directory (create the file if it doesn't exist), using your Gitea username and the token:

machine gitea-direct.abair.ie
login <your-gitea-username>
password <your-token>

Then make it readable only by you:

chmod 600 ~/.netrc

Keep the token out of repos

Don't put the token in a repo's .git/config, in a remote URL, or in any committed file. ~/.netrc keeps it out of git entirely. If a token leaks, delete it in Settings → Applications and generate a new one.

Uploading files to a repo

3. Clone the repo and tell Git LFS which files are large

Clone over SSH, always using gitea-direct.abair.ie:

git clone git@gitea-direct.abair.ie:ABAIR/<repo>.git
cd <repo>

Tell Git LFS which file types to handle. Do this before adding the files:

git lfs track "*.nemo" "*.bin" "*.arpa"    # use the extensions your files have
git add .gitattributes

This writes the patterns to .gitattributes, which is committed with the repo. If the repo already has a .gitattributes covering your file types, skip this step.

4. Point Git LFS at gitea-direct

Run this inside the repo, replacing <repo>:

git config lfs.url https://gitea-direct.abair.ie/ABAIR/<repo>.git/info/lfs

This is saved in the repo's local .git/config only, so run it again in every new clone of a repo you upload large files to.

This also applies to downloads in this clone

With this setting, Git LFS uses gitea-direct for downloads too, so git pull of large files in this clone only works from the TCD network or VPN. To pull from elsewhere, remove the setting with git config --unset lfs.url, or keep a separate clone without it.

5. Add, commit and push

Make sure you're on the TCD network or VPN, then:

git add <your files>
git commit -m "Add <model name>"
git push origin main

Git LFS prints an upload progress line for each large file. Large models can take several minutes.

6. Check it worked

git lfs ls-files

Every large file should be listed. In Gitea's web interface, opening the file should show Stored with Git LFS.

Downloading files

Anyone with read access can clone or pull from anywhere. Large files download over gitea.abair.ie, and Cloudflare's limit only applies to uploads:

git clone git@gitea-direct.abair.ie:ABAIR/<repo>.git

To download a single file without cloning, for example in a Dockerfile or script, use the /media/ URL, not /raw/:

https://gitea.abair.ie/ABAIR/<repo>/media/branch/main/<file>

/raw/ returns a small text "pointer" file instead of the real file for anything stored in Git LFS.

Troubleshooting

What you see	Cause	Fix
HTTP 413 or Request Entity Too Large, and the push retries several times	Git LFS is uploading to gitea.abair.ie through Cloudflare	Run step 4 in this repo
The upload hangs, then fails with a timeout or connection timed out	You're not on a TCD address, so gitea-direct refuses HTTPS	Connect to the TCD network or VPN and push again
401 Unauthorized, or git asks for a username and password	No token, or a wrong or expired token, for gitea-direct	Check ~/.netrc (step 2). Make sure the token has repository: Read and Write
403 Forbidden	Your account can't write to this repo	Ask an admin for write access
Files show up in Gitea as about 130 bytes of text starting version https://git-lfs...	The pointer was pushed but the file wasn't uploaded	Run git lfs push --all origin from the TCD network, with step 4 done
git lfs ls-files doesn't list a large file	It was committed before git lfs track, so it went into plain git	Ask an admin. Plain-git files this size will be rejected or bloat the repo, and fixing it means rewriting the commit

Avoid the web upload button for models

Gitea's Upload File button in the browser has a 50MB limit per file and goes through Cloudflare. Use the git commands above for anything large, and don't split models into parts to get around the limit.

If none of these match, send an admin the full output of the failing command. Running it with GIT_TRACE=1 GIT_CURL_VERBOSE=1 in front gives more detail, but delete any Authorization: lines from that output before sharing it.

For how gitea-direct and the firewall are set up, see Secops → Gitea.